Multi-scale hybrid vision transformer and Sinkhorn tokenizer for sewer defect classification

A crucial part of image classification consists capturing non-local spatial semantics content. This paper describes the multi-scale hybrid vision transformer (MSHViT), an extension classical convolutional neural network (CNN) backbone, for multi-label sewer defect classification. To better model in images, features are aggregated at different scales non-locally through use a lightweight transformer, and smaller set tokens was produced novel Sinkhorn clustering-based tokenizer using distinct cluster centers. The proposed MSHViT were evaluated on Sewer-ML dataset, showing consistent performance improvements up to 2.53 percentage points.